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Abstract. We develop an analytical model of information dissemination 
for a gossiping protocol that combines both pull and push approaches. 
With this model we analyse how fast an item is replicated through a 
network, and how fast the item spreads in the network, and how fast 
the item covers the network. We also determine the optimal size of the 
exchange buffer, to obtain fast replication. Our results are confirmed by 
large-scale simulation experiments. 

1 Introduction 

Today, large-scale distributed systems consisting of thousands of nodes are com- 
monplace, due to the wide availability of high-performance and low-cost devices. 
Such systems are highly dynamic in the sense that nodes are continuously in 
flux, with new nodes joining and existing nodes leaving. 

In practice, large-scale systems are often emulated to discover correlations 
between design parameters and observed behaviour. Such experimental results 
provide essential data on system behaviour. However, they usually show only 
behaviour of a particular implementation, and can be time consuming. More- 
over, in general experiments do not give a good understanding of the emergent 
behaviour of the system, and into how parameter settings influence the extra- 
functional properties of the system. As a result, it is very difficult to predict 
what the effects of certain design decisions are, as it is practically infeasible to 
explore the full range of input data. A challenge is to develop analytical models 
that capture (part of) the behaviour of a system, and then subsequently optimize 
design parameters following an analytical rather than an experimental approach. 

We are interested in developing and validating analytical models for gossip- 
based systems (cf. [1]). These systems rely on epidemic techniques for the com- 
munication and exchange of information. These communication protocols, while 
having simple specifications, show complex and often unexpected behaviour 
when executed on a large scale. Our analytical models of gossip protocols need 
to be realistic, yet, sufficiently abstract to allow for easy prediction of systems 
behaviour. By 'realistic' we mean that they can be applied to large-scale systems 
and can capture functional and extra-functional behaviour such as replication, 
coverage, convergence, and other system dynamics (see [2]). Such models are 



amenable for mathematical analysis, to make precise predictions. Furthermore, 
we will exploit the fact that because an analytical model presents an abstrac- 
tion of the original protocol, a simulation of the model tends to be much more 
efficient (in computation time and memory consumption) than a simulation of 
an implementation of this protocol. 

In this paper, we develop an analytical model of a shuffle protocol from [3], 
which was developed to disseminate data items to a collection of wireless devices, 
in a decentralized fashion. A decentralized solution considerably decreases the 
probability of information loss or unavailability that may occur due to a single 
point of failure, or high latency due to the overload of a node. Nodes executing 
the protocol periodically contact each other, according to some probability dis- 
tribution, and exchange data items. Concisely, a node initiates a contact with its 
random neighbour, pulls a random subset of items from the contacted node, si- 
multaneously pushing its own random subset of items. This push/pull approach 
has a better performance than a pure push or pull approach [4, 5] . The amount of 
information exchanged during each contact between two communicating nodes 
is limited. Replication ensures the availability of the data items even in the face 
of dynamic behaviour, which is characteristic of wireless environments. Thus, 
nodes not only conserve the data collectively stored in the network, but also 
relocate it in a random fashion; nodes will eventually see all data items. 

The central point of our study is a rigorous probabilistic analysis of informa- 
tion dissemination in a large-scale network using the aforementioned protocol. 
The behaviour of the protocol is modelled on an abstract level as pairwise node 
interactions. When two neighbouring nodes interact with each other (gossip), 
they may undergo a state transition (exchange items) with a certain probability. 
The transition probabilities depend on the probability that a given item in a 
node's cache has been replaced by another item after the shuffle. We calculated 
accurate values for these probabilities, yielding a rather complicated expression. 
We also determined a close approximation that is expressed by a much simpler 
formula, as well as a correction factor for this approximation, allowing for precise 
error estimations. Thus we obtain a better understanding of the emergent be- 
haviour of the protocol, and how parameter settings influence its extra-functional 
behaviour. 

We investigated two properties characterizing the protocol, namely, the num- 
ber of replicas of a given item in the network at a certain moment in time (repli- 
cation), and the number of nodes that have 'seen' this item over time (cover- 
age). Using the values of the transition probabilities, we determined the optimal 
number of items to exchange per gossip, for a fast convergence of coverage and 
replication. Moreover, we determined formulas that capture the dissemination of 
an item in a fully connected network. All our modelling and analysis results are 
confirmed by large-scale simulations, in which simulations based on our analyt- 
ical models are compared with running the actual protocol. To the best of our 
knowledge, we are the first to develop an accurate, realistic formal model that 
can be used to optimally design and fine-tune a given gossip protocol. In this 
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sense, our main contribution is demonstrating the feasibility of a model-driven 
approach to developing real- world gossip protocols. 

The paper is structured as follows. The remainder of this introduction dis- 
cusses related work. Section 2 explains the shuffle protocol. In Section 3 the 
analytical model is developed. Section 4 discusses the results of our experimen- 
tal evaluations. Section 5 presents a round-based perspective of replication and 
coverage. And Section 6 contains the conclusions. 

Related work 

Two areas of research are relevant to our paper: rigorous analysis of gossip (and 
related) protocols, and results from mathematical theory of epidemics [6, 7]. The 
results from epidemics are often used in the analysis of gossip protocols [8]. We 
restrict our overview to the most relevant publications from the area of gossip 
protocols. 

Several works have focused on gossip-based membership management proto- 
cols. 

Allavena et al. [9] proposed a gossip-based membership management protocol 
and analysed the evolution of the number of links between two nodes executing 
the protocol. The states of the associated Markov chain are the number of links 
between pairs of nodes. From the designed Markov chain they calculated the 
expected time until a network partition occurs. This case study also includes a 
model of the system under churn. A goal of that paper is to show the effect of 
mixing both pull and push approaches. 

Eugster ct al. [10] presented a lightweight probabilistic broadcast algorithm, 
and analysed the evolution of processes that gossip one message. The states of 
the associated Markov chain are the number of processes that propagate one 
gossip message. From the designed Markov chain, the authors computed the 
distribution of the gossiping nodes. Their analysis has shown that the expected 
number of rounds to propagate the message to the entire system does not depend 
on the out-degree of nodes. These results are based on the analysis assumption 
that the individual out-degrees arc uniform. However, this simplification has 
shown to be valid only for small systems (cf. [4]). 

Bonnet [11] studied the evolution of the in-degree distribution of nodes exe- 
cuting the Cyclon protocol [12]. The states of the associated Markov chain are 
the fraction of nodes with a specific in-degree distribution. From the designed 
Markov chain the author determined the distribution to which the protocol con- 
verges. 

There are a number of theoretical results on gossip protocols, targeted to a 
distributed aggregation. 

Boyd et al. [13] studied the averaging problem and analysed a gossip protocol 
in which nodes compute the average of their local measurements. The Markov 
chain is defined by a weighted random walk on the graph. Every time step, a 
pair of nodes (connected by an edge) communicates with a transition probability, 
and sets their values equal to the average of their current values. A state of the 
associated Markov chain is a vector of values at the end of the time step. The 
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authors considered the optimization of the neighbour selection probabilities for 
each node, to find the fastest-mixing Markov chain (for fast convergence of the 
algorithm) on the graph. 

Jelasity et al. [14] proposed a solution for aggregation in large dynamic net- 
works, supported by a performance analysis of the protocol. A state of the system 
is represented by a vector, the elements of which correspond to the values at the 
nodes, a target value of the protocol calculated from the vector elements, and a 
measure of homogeneity characterizing the quality of local approximations. The 
vector evolves at every step of the system according to some distribution. In the 
analysis, the authors considered different strategies (e.g., neighbour selection) 
to optimize the protocol implementation, and calculated the expected values for 
the abovementioned protocol parameters. 

Deb et al. [15] studied the adaptation of random network coding to gossip 
protocols. The authors analysed the expected time and message complexity of 
two gossip protocols for message transmission with pure push and pure pull 
communication models. 

2 A Gossip-based Protocol for Wireless Networks 

This section describes the shuffle protocol introduced in [3] . It is a gossip protocol 
to disseminate small data items of general interest to a collection of wireless 
devices. The protocol relies on replication to ensure the availability of data items 
in the face of dynamic behaviour, which is characteristic of wireless environments. 

The system consists of a collection of wireless nodes, each of which contributes 
a limited amount of storage space (which we will refer to as the node's cache) 
to store data items. The nodes periodically swap (shuffle) data items from their 
cache with a randomly chosen neighbour. In this way, nodes update their caches 
on a regular basis, allowing nodes to gradually discover new items as they are 
disseminated through the network. 

Items can be published by any user of the system, and arc propagated through 
the network. While an item is a piece of information, a copy is the representation 
of the item in the network, and for each item several copies may exist. As items 
are gossiped between neighbouring nodes, replication may occur when a node has 
available storage space to keep a copy of an item it just gossiped to a neighbour. 

2.1 Protocol assumptions 

All nodes have a common agreement on the frequency of gossiping. However, 
there is no agreement on when to gossip. 

In terms of storage space, we assume that all nodes dedicate the same amount 
of storage space to keep items locally, and that all items are of the same size. 
Therefore, we say that each node has a cache size of c. When shuffling, each 
node sends a fixed number s of the c items in the cache. 

The gossip exchange is performed as an atomic procedure, meaning that 
once a node initiates an exchange with another node, these pair of nodes cannot 
become involved in another exchange until the current exchange is finished. 
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2.2 Description 



Nodes executing the shuffle protocol initiate a shuffle periodically. In order to 
execute the protocol, the initiating node needs to contact a gossiping partner. 
We describe the protocol from the point of view of each participating node. We 
refer to [3] for a more detailed description. 

Node A initiates the shuffle by executing the following steps: 

1. picks a neighbouring node B at random; 

2. selects randomly s items from the local cache, and sends a copy of these 
items to B; 

3. receives s items from the local cache of B; 

4. checks whether any of the received items are already in its cache; if so, these 
received items are eliminated; 

5. adds the rest of the received items to the local cache; if the total number of 
items exceeds cache size c, removes items among the ones that were sent by 
A to B, but not those that were also received by A from B, until the cache 
contains c items. 

In response to being contacted by A, node B executes the following steps: 

1. receives s items from the local cache of A; 

2. selects randomly s items from its local cache, and sends a copy of these items 
to A; 

3. checks whether any of the received items are already in its cache; if so, these 
received items are eliminated; 

4. adds the rest of the received items to the local cache; if the total number of 
items exceeds cache size c, removes items among the ones that were sent by 
B to A, but not those that were also received by B from A, until the cache 
contains c items. 

According to the protocol, each node agrees to keep the items received from 
a neighbour. Given the limited storage space available in each node, keeping the 
items received during an exchange implies discarding some items that the node 
has in its cache. By picking the items to be discarded from the ones that have 
been sent to the neighbour, the conservation of data in the network is ensured. 

2.3 Properties 

We are interested in characteristics of the dissemination of data items when 
the protocol is executed at a large scale, i.e. with a large set of nodes. For this 
reason, we focus on two properties that can be observed in large deployments: i) 
the number of replicas of an item in the network, and ii) the coverage achieved 
by an item over time. 

Replication This property is defined as the fraction of nodes that hold a copy 
of a generic item d in their cache, at a given moment. After an item is introduced 
into the network, with every shuffle involving a node that has the item in its 
cache, there is a chance that a new copy of the item will be created, or that 
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the item will be discarded. As a result, with every passing round the number 
of copies in the network for a particular item fluctuates. Given that the storage 
space at the nodes is limited, items are in constant competition to place copies 
in the network. Since competition is fair (all items have the same chance of 
being replicated or discarded), eventually the storage capacity is evenly divided 
between the existing items. To be more precise, consider a network of N nodes, 
in which n different items have been published in total. Since there are N-c cache 
entries in the network in total, the average number of copies that an individual 
item has in the network will converge to So replication will converge to ^. 

Coverage This property is defined as the fraction of nodes in the network 
that have seen a generic item d since it was introduced into the network. As 
explained earlier, several copies of an item are generated after the item is first 
published. Due to the periodic nature of the protocol, these copies continually 
move through the network. This results in nodes discovering item d over several 
rounds. With each passing round, more nodes will have seen d. Eventually, d will 
have been seen by all nodes (i.e., the coverage is equal to 1). The speed at which 
the coverage grows is influenced by several factors (as will be explained later on) 
including the number of different items in the network (i.e. competition), cache 
size, and the size of the exchange buffer. 

3 An Analytical Model of Information Dissemination 

We analyse dissemination of a generic item d in a network in which the nodes 
execute the shuffling protocol. 

3.1 Probabilities of state transitions 

We present a model of the shuffle protocol 
that captures the presence or absence of a 
generic item d after shuffling of two nodes 
A and B. There are four possible states of 
the caches of A and B before the shuffle: 
both hold d, either A's or B's cache holds 
d, or neither cache holds d. 

We use the notation P{a2b 2 \aibi) for 
the probability that from state di&i af- 
ter a shuffle we get to state 0262, with 
£Jj,6j <G {0,1}. The indices a\, a 2 and 61, 
62 indicate the presence (if equal to 1) or 
the absence (if equal to 0) of a generic 
item d in the cache of an initiator A and 
the contacted node B, respectively. For 
example, P(01|10) means that node A had 
d before the shuffle, which then moved to 



P(00|00) 



P(01|01) 




P(01|ll) 



P(10|10) f(n|ii) 
Fig. 1. Symbolic representation for 
caches of gossiping nodes. 
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the cache of B, afterwards. Due to the symmetry of information exchange be- 
tween nodes A and B in the shuffle protocol, P(a2& 2 |aifri) = P (&2<i2|&iGn)- 

Fig. 1 depicts all possible outcomes for the caches of gossiping nodes as a 
state transition diagram. If before the exchange A and B do not have d {a\b\ = 
00), then clearly after the exchange A and B still do not have d (0262 = 00). 
Otherwise, if A or B has d (ai = 1 V b\ = 1), the shuffle protocol guarantees 
that after the exchange A or B still has d (a 2 = 1 V b 2 = 1). Therefore, the state 
(— , — ) has a self-transition, and no other outgoing or incoming transitions. 

We determine values for all probabilities P(a 2 & 2 |aifri). They are expressed 
in terms of probabilities P se iect and Pdrop ■ The probability P se iect expresses the 
chance of an item to be selected by a node from its local cache when engaged in 
an exchange. The probability Pdrop represents a probability that an item which 
can be overwritten (meaning it is in the exchange buffer of its node, but not of 
the other node in the shuffle) is indeed overwritten by an item received by its 
node in the shuffle. Due to the symmetry of the protocol, these probabilities are 
the same for both initiating and contacted nodes. In Sec. 3.2, we will calculate 
Pseiect and P drop . We write P^seiect for 1 - P se i ect and P^ drop for 1 - Pdrop- 

Scenario 1 (a\bi = 00) Before shuffling, neither node A nor node B have d in 
their cache. 

0262 = 00: neither node A nor node B have item d after a shuffle because neither 

of them had it in the caches before the shuffle: P(00|00) = 1 
0262 € {01, 10, 11}: cannot occur, because none of the nodes have item d. 

Scenario 2 {a\b\ =01) Before shuffling, a copy of d is only in the cache of node 
B. 

0262 = 01: node A does not have d because node B had d but did not select it 
(to send) and, thus, B did not overwrite d, i.e. the probability is P(01|01) = 

P—iselect 

(Z2&2 = 10: only node A has d because node B selected d and dropped it; that 
is, the probability is P(10|01) = Pseiect ■ Pdrop 

a2&2 = 11: both nodes A and B have a copy of d because node B selected d and 
kept it; that is, P(ll|01) = P se iect ■ P^drop 

(Z2&2 = 00: cannot occur as completely discarding d is not possible in the pro- 
tocol; that is, if cither nodes send an item, its partner keeps this copy as 
well, and if an item is not among the selected for a shuffle, the item is not 
replaced by another one (see Sec. 2.2). 

Scenario 3 {a\bi = 10) Before shuffling, d is only in the cache of node A. Due 
to the symmetry of nodes A and B, this scenario is symmetric to the previous 
one with P(a 2 6 2 |10) = P(6 2 a 2 |01). 

Scenario 4 (a\bi = 11) Before shuffling, d is in the cache of node A as well as 
in the cache of node B. 

(Z2&2 = 01: only node B has d because node A selected d and dropped it and 
node B did not select d; that is, P(01|ll) = P se iect ■ Pdrop ■ P^seiect 
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a2&2 = 10: this outcome is symmetric to the previous one: P(10|ll) = P^seiect • 

Pselect ' Pdrop 

= 11: after the shuffle both nodes A and B have d, because: 

- nodes A and B had d but both did not select it, i.e. P-, S eiect • P^seiecu 

- both nodes A and B selected d (thus, both kept it), i.e. P se lect ■ Pselect, 

- node A selected d and kept it and node B did not select d: P se iect • P-.dro P • 

P— 'Select , 

- symmetric case with the previous one: Prelect • Pselect • P^drop- 

Thus, P(ll|ll) = P^selecf P-.select+ Pselecf Pselect + 2' Pselecf P^selecf P-.drop 

a2&2 = 00: cannot occur, discarding of an item is not permitted by the protocol 
(see Sec. 2.2). 

3.2 Probabilities of selecting and dropping an item 

The following analysis assumes that all node caches are full (that is, the network 
is already running for a while). Moreover, we assume a uniform distribution of 
items over the network; this assumption is supported by experiments in [3,4]. 

Consider nodes A and B engaged in a shuffle, and let B receive the exchange 
buffer Sa from A. Let k be the number of duplicates (see Fig. 2), i.e. the items 
of an intersection of the node cache Cb and the exchange buffer of its gossiping 
partner Sa (i.e. Sa H Cb)- Recall from Sec. 2.1 that Ca and Cb contain the 
same number of items for all A and B, and likewise for Sa and Sb', we use c and 
s for these values. The total number of different items in the network is denoted 
as n. 




Fig. 2. k items in Sa PI Cb Fig. 3. s items in Sa PI Sb 



The probability of selecting an item d in the cache is the probability of a 
single selection trial (i.e. -) times the number of selections (i.e. s): Pselect = f • 
Thus, the probability that an item d in the cache is not selected is: P^ se iect = 

1 p c—s 

± 1 select — c • 

Consider Figs. 2 and 3. The shuffle protocol demands that all items in Sa 
are kept in Cb after the shuffle. This implies that: a) all items in Sa\Cb will 
overwrite items in Sb C Cb, and b) all items in Sa n Cb are kept in Cb. Thus, 
the probability that an item from Sb will be overwritten is determined by the 



8 



probability that an item from Sa is in Cb, but not in Sb- Namely, the items 
in Sb\Sa provide a space in the cache for items from Sa\Cb- We would like to 
express the probability Pdrop of a selected item d in Sb\Sa (or Sa\Sb) to be 
overwritten by another item in Cb (or Ca)- Due to symmetry, this probability is 
the same for A and B; therefore, we only calculate the probability that an item 
in Sb\Sa is dropped from Cb- The expected value of this probability depends 
on how many duplicates a node receives from its gossiping partner: 



E[P, 



drop] 



k=0 



E (^™ p nCflM • P\S A nc B \= k ) otherwise 

-{s+c)—n 



where P\s A nc B \=k is the probability of having exactly k items in Sa n Cb, and 

P drop° Bl=k is the probability that an item in Sb\Sa is dropped from Cb given 
k duplicates in Sa^Cb- The case distinction is because if s + c > n, then clearly 
there are at least (s + c) — n items in Sa H Cb- 

From the (™) possible sets Sa, we compute how many have k items in common 
with Cb- Firstly, there are (£) ways to choose k such items in Cb- Secondly, there 
are ways to choose the remaining s — k items outside Cb- So in total, 

(fc) ' ("Zfc) possible sets Sa have k items in common with Cb- Hence, under the 
assumption of a uniform distribution of the data items over the caches of the 

nodes, 1 P\s A nc B \=k = (fc)W' The ex P ected value 01 P lro p nCBl=k i^ 



w r-p\S A C\C B \=k, 



C) 

Y,P ] l: p nSB] ^-P\s A ns B \=s ifs + fc^ 

E P i™r Bl ^%.n5 B |=s oth ™ e 

i s=(s+fe)-c 



where s is the number of items in Sa H (see Fig. 3). The case distinction is 
because if s + k > c (with k the number of items in Sa n Cb), then clearly there 
are at least (s + k) — c items in Sa H Sb- 

Among the s items in Sb, there are s items also in S^, and thus only the 
s — s items in Sb\Sa can be dropped from Cb. -Pifo p nSB ' _S is the probability 
that an item in Sb\Sa is dropped from Cb, given s items in Sa n 5b: 

p |s A ns B |=? = |0 if s = s 
drop |2=£ otherwise 

^ s — s 

P\S A ns B \=s is the probability of having exactly s items in Sa^Sb- E[P\S A nS B \=s] = 
(?) T^F • ^ nc intuition behind this expected value of -FlsAns B |=s is similar to 

1 Here we use a generalization of the usual definition of binomial coefficients to negative 
integers. That is, for all m and I > 0, ('") = (" m | i " 1 ) (cf. [16]) 
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the one of P\s A nc B \=k- From the (£) possible sets Sa, we compute how many 
have s items in common with Sb- That is, there are (i) ways to choose s items 
in Sb, and (£Z--) ways to choose the remaining k — s items outside Sb- 

Let's assume 2s < c < n — s (because then s + c < n and s + k < 2s < c). 
Then, substituting in the expression for E[P drop ] in case s + c < n, and noting 

that in the summand k = s the factor P d ro p nSB ^~ S is equal to zero, we get: 
prp i ( c \ (s-fc) s — k fs\ (fc_|) 

n - c ^ /(n - c) - 1\ ^ (fc-s) (?) m 

H e^\(*-*)-v^ 

W fc=0 vv 7 7 s=0 

The probability of keeping an item d in Sb\Sa C C_b can be expressed as 

^-i drop 1 Pdrop • 



3.3 Simplification of Pdrop 

In order to gain a clearer insight into the emergent behaviour of the gossiping 
protocol we make an effort to simplify the formula for the probability Pdrop of 
an item in Sb\Sa to be dropped from Cb after a shuffle. Therefore, we re- 
examine the relationships between the k duplicates received from a neighbour, 
the s items of the overlap Sa n Sb, and Pdrop- Let's estimate -P^ p nC ' B ' _ ' c by 
considering each item from Sa separately, and calculating the probability that 
the item is a duplicate (i.e., is also in Cb)- The probability of an item from Sa 
to be a duplicate (also present in Cb) is -■ In view of the uniform distribution 
of items over the network, the items in a node's cache are a random sample from 
the universe of n data items; so all items in Sa have the same chance to be a 
duplicate. Thus, the expected number of items in Sa n Cb can be estimated by 
E[k] — s ■ —. And the expected number of items in Sa H Sb can be estimated 
by E\s\ = k ■ |, because only the k items in Sa^Cb may end up in SaC^ Cb', 
| captures the probability that an item from Cb is also selected to be in Sb- It 
follows that the probability of an item in Sb\Sa to be dropped from Cb after 
the shuffle is E [Pdrop] = §5§ = S !^ZZ = fjEf • The complementary probability 

of keeping an item is E[P-, d rop] — 1 — ^Ef — ^Ef • These estimates are valid for 
general s < c < n. 

Substituting the expressions for P se iect and the simplified Pdrop into the for- 
mulas for the transition probabilities in Fig. 1, we obtain: 

P(01|01) = P(10|10) = *=* P(01|ll) = P(10|ll) = f^^Ef 

P(10|01) = P(01|10) = f 2Ef ^(U|U) = 1 - 2f ^^Ef 

P(ll|01) = P(ll|10) = ffEf 

In order to verify the accuracy of the proposed simplification for E[P dr op}, we 
compare the simplification and the accurate formula (1) for different values of 
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n. We plot the difference of the accurate Pdrop and the simplification, for cache 
sizes c = 250 and c = 500 (Fig. 4). 




3.4 Correction factor 

We now examine how closely the simplified formula E [Pdrop] = ^E^ (here re- 
ferred as S(n,c,s)) approximates formula (1) (here referred as E(n,c,s)). We 
compared the difference between these two formulas using an implementation 
on the basis of common fractions, which provides loss-less calculation [17]. We 
observed that the inverse of the difference of the inverse values of both formulas, 
i.e. e CiS (n) = (E(n, c, s) _1 — S(n, c, , exhibits a certain pattern for differ- 

ent values of n, c and s. For s = 1, E(n, c, 1) = whereas S(n, c, 1) = ^5f • 
We then investigate the correction factor 9 in Efn.c.s) = , n ~\. a . Thus, for 
s = 1 we have 9 = 1. Yet, for s > 1 the situation turned out to be more 
complicated. For s — 2, we got 64,2(7) — 64,2(6) = 3.5, 64,2(8) — e4, 2 (7) = 4, 
e4,2(9) — 64,2(8) = 4.5, and etc. Therefore we calculated the first, the second and 
other (forward) differences 2 over n. We recognized that the s-th difference of the 
function e c , s (n) is always -. Moreover, at the point n = the 1st, . . . , s-th dif- 
ferences of the function e c . s exhibit a pattern similar to the Pascal triangle [19]; 
i.e. for d > 1 the d-th difference is: (A d e c . s )(0) = (s-i\ (assuming (?) = 0, 

whenever b > a). Knowing the initial difference at the point n — 0, we were 
able to use the Newton forward difference equation [18] to derive the following 
formula for n > 0: E [Pdrop] — — "~ c 1 , where 



\ " {'d) = (") \ " 1 (2) 



d=0 



2 A forward difference of discrete function / : Z — + Z is a function A f : 7L — * Z with 
4f(n)=/(n + l)-/(n) (cf. [18]). 
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In this equation the sum is finite because due to the observation that the s-th 
difference is constant =, all higher differences are 0. 

Extensive experiments with Mathematica and Matlab indicate that r^j^r 

and formula (1) coincide. We can also see from Fig. 4 that the correction factor 
is small. 



3.5 Optimal size for the exchange buffer 



70 



55 



We study what is the optimal value 
for fast convergence of replication 
and coverage with respect to an 
item d. Since d is introduced at 
only one node in the network, one 
needs to optimize the chance that 
an item is duplicated. That is, the 
probabilities P(ll|01) and P(ll|10) 
should be optimized (then P(01|ll) 
and P(10|ll) are optimized as well, 
intuitively because for each dupli- 
cated item in a shuffle, another item 
must be dropped). These probabil- 
ities both equal ~fErl we compute 
when the s-derivative of this formula is zero. This yields the equation s 2 — 2ns + 
nc — 0; taking into the account that s < n, the only solution of this equation is 
s = n — yj n(n — c) . We conclude that this is the optimal value for s to obtain 
fast convergence of replication and coverage. This will also be confirmed by the 
experiments and analyses in the following sections. 
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Fig. 5. Optimal value of exchange buffer 
size, depending on n. 



4 Experimental Evaluation 

In order to test the validity of the analytical model of information spread under 
the shuffle protocol presented in the previous section, we followed an experi- 
mental approach. We compared properties observed while running the shuffle 
protocol in a large-scale deployment with simulations of the model under the 
same conditions. These experiments show that the analytical model indeed cap- 
tures information spread of the shuffle protocol. We note that a simulation of 
the analytical model is much more efficient (in computation time and memory 
consumption) than a simulation of the implementation of the shuffle protocol. 

The experiments simulate the case where a new item d is introduced at one 
node in a network, in which all caches are full and uniformly populated by 
n = 500 items. They were performed on a network of N = 2500 nodes, arranged 
in a square grid topology (50x50), where each node can communicate only with 
its four immediate neighbours (to the North, South, East and West). 
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This configuration of nodes is arbitrary, we only require a large number of 
nodes for the observation of emergent behaviour. Our aim is to validate the cor- 
rectness of our analytical model, not to test the endless possibilities of network 
configurations. The model and the shuffle protocol do not make any assumptions 
about the network. The network configuration is provided by the simulation en- 
vironment and can easily be changed into something different, e.g. other network 
topology. For this reason, we have chosen this large grid for testing, although 
other configurations could have been possible. 

Each node has a cache size of c = 100, and sends s items when gossiping. 
In each round, every node randomly selects one of its neighbours, and updates 
its state according to the transition probabilities introduced before (Fig. 1). 
This mimics (the probabilities of) an actual exchange of items between a pair of 
nodes according to the shuffle protocol. While in the protocol, this results in both 
nodes updating the contents of their caches, in a simulation using the analytical 
model, updating the state of a node refers to updating only one variable: whether 
the node is in possession of the item d or not. In the experiments, after each 
gossiping round, we measured the total number of occurrences of d in the network 
(replication), and how many nodes in total have seen d (coverage); see Sec. 2.3. 

In order to fill the caches of the nodes with a random selection of items, 
the measurements are initiated after 1000 rounds of gossiping. In other words, 
500 different items are inserted at the beginning of the simulation, and shuffled 
for 1000 rounds. During this time, items are replicated and the replicas fill the 
caches of all nodes. At round 1000, a copy of the fresh item d is inserted at a 
random location, and its spread through the network is tracked over the next 
2000 rounds. 




Fig. 6. The shuffle protocol (left) and the model (right), for N = 2500, n = 500, 
c = 100 and different values of s. 



13 



Fig. 6 shows the behaviour of both the shuffle protocol and the analytical 
model in terms of replication and coverage of d, for various values of s. Each 
curve in the graphs represents the average and standard deviation calculated over 
100 runs. The experiments with the model calculate Pdrop using the simplified 
formula described in Sec. 3.3. It can be observed very clearly that the results 
obtained from the model (right) resemble closely the ones from executing the 
protocol (left). 

We note that in all cases, the network converges to a situation in which there 
are 500 copies of d, meaning that replication is = 0.2; this agrees with the 
fact that ^ = -g^y = 0.2. Moreover, our experiments show that replication and 
coverage display the fastest convergence when s = 50; this agrees with the fact 
that n - yjn{n- c) = 500 - V500-400 w 50 (cf. Sec. 3.5). 

5 Round-based Modelling of Protocol Properties 

In this section we exploit the analytical model of information dissemination to 
perform a mathematical analysis of replication and coverage with regard to the 
shuffle protocol. For the particular case of a network with full connectivity, where 
a node can gossip with any other node in the network, we can find explicit ex- 
pressions for the dissemination of a generic item d in terms of the probabilities 
presented in Sec. 3. We construct two differential equations that capture repli- 
cation and coverage of item d from a round-based perspective. The advantage of 
this approach is that we can determine the long-term behaviour of the system 
as a function of the parameters. 

5.1 Replication 

One node introduces a new item d into the network at time t — 0, by placing d 
into its cache. From that moment on, d is replicated as a consequence of gossiping 
among nodes. 

Let x(t) represent the percentage of nodes in the network that have d in their 
cache at time t, where each gossip round takes one time unit. The variation in 
x per time unit ^ can be derived based on the probability that an item d will 
replicate or disappear after an exchange between two nodes, where at least one 
of the nodes has d in its cache: 

dx 

— = [P(H|10) + P(H|01)] -{l-x)-x- [P(10|ll) + P(01|ll)] -x-x 

The first term represents duplication of d when a node that has d in its cache 
initiates the shuffle, and contacts a node that does not have the item. The 
second term represents the opposite situation, when a node that does not have 
the item d initiates a shuffle with a node that has d. The third and fourth 
term in the equation represent the cases where both gossiping nodes have d in 
their cache, and after the exchange only one copy of d remains. Substituting 
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P(ll|10) = P(ll|01) = |^E| and P(10|ll) = P(01|ll) = § ^Ef we obtain 



dx 



2- 



x- (1 



x) 



(3) 



The solution of this equation, taking into account that x(0) 
number of nodes in the network, is 



^ with N the 



x(t) 



(N 



(4) 



where a denotes 2|^5f- By imposing stationarity, i.e. 4| = 0, we find the 
stationary solution £ Hence, this calculation confirms the observation in Sec. 2.3 
that the network converges to a situation in which replication of d is — . 
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Fig. 7. Percentage of nodes in the network with a replica of item d in their cache, for 
N = 2500, c = 100, s = 50, and n = 500, n = 1000 or n = 2000. 



We evaluate the accuracy of x(t) as a representation of the fraction of nodes 
carrying a replica of d, by running a series of experiments where N = 2500 nodes 
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execute the shuffle protocol, and their caches are monitored for the presence of 
d. Unlike the experiments in Sec. 4, we assume full connectivity; that is, for 
each node, all other nodes are within reach. After 1000 rounds, where items 
are disseminated and replicated, a new item d is inserted at a random node, at 
time t = 0. We track the number of replicas of d for the next 1000 rounds. The 
experiment is repeated 100 times and the results are averaged. These simulation 
results (average and standard deviation) for the protocol, together with x(t), 
are presented in Fig. 7. This figure shows the same initial increase in replicas 
after d has been inserted, and in all cases the steady state reaches precisely the 
expected value - predicted from the stationary solution. 

We repeat the calculation from Sec. 3.5, but now against x(t), to deter- 
mine which size of the exchange buffer yields the fastest convergence to the 
steady-state for both replication and coverage. That is, we search for the s that 
maximizes the value of x(t). We first compute the derivative of x(t) with respect 
to s (z(t,s)), and then derive the value of s that maximizes x(t), by taking 

z (. m \ - 9x\ _ q. U \ _ dx _ 2e kt (cN-n)(cn+ S (-2n+s))t u l _ ns c^s 

Let z(t 7 s) = 0. Fo r t > 0, cn = s(2n — s). Solving this equation we get 
s = n ± \/n(n — c). Taking into the account that s < n, the only solution is 
s = n — \Jn{n — c). So this coincides with the optimal exchange buffer size 
found in Sec. 3.5. 



5.2 Coverage 

We use the term coverage to denote the percentage of nodes in the network that 
have seen item d from the moment it was introduced into the network. Let y{t) 
represent the coverage of d at time t. The variation in coverage per time unit, 
is determined by the fraction of nodes that have not seen d, 1 — y, that interacts 
with nodes that have d in their cache, x. Let * e {0, 1}, then: 

^=P(l*|01)-P(*l|*l)-(l-y)-a; 

+ P(l*|01)-P(l*|l*)-a;-(l-j/) (5) 

The first term is represents increased coverage due to nodes discovering d after 
interacting with nodes that have d in their cache. This can occur when a node 
initiates the exchange (P(l*|01)), or when the node is contacted (P(*l|10)). 
The second part of these terms represents the case when a node discovers and 
does not give away its copy of d within the same round to another node. This 
is because coverage is only measured at the end of a gossiping round, meaning 
that a node that sees item d for the first time, and drops it in the same round, 
is considered not to have seen item d yet. 3 Since nodes shuffle, on average, twice 
per round (once when they initiate the shuffle and again if they are contacted 
by a neighbour), this could occur under two scenarios: i) the node acquired d 

3 The reason for this is that the application has an opportunity to read from the 
lower- level cache only once every round. 
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by initiating an exchange with a node that had d (P(l*|01)) and next lost its 
copy of d when shuffling with a node that contacted it (P(*l|*l)), or ii) the 
node was first contacted by a node that sent a copy of d (P(*l|10)) and next 
initiated a shuffle and gave away its copy of d (P(l*|l*)). Note that P(*l|*l) = 
f — P(01|f0), as the probability P(01|f0) is the only transition probability that 
does not match the pattern P(*l|*l). Due to the symmetry of both gossiping 
nodes, P(*l|*l) = P(l*|l*). 

Substituting (4) into (5), we obtain 




1000 



rounds 
2000 items 



Fig. 8. Percentage of nodes in the network that have already seen a replica of item d, 
for N = 2500, c = 100, s = 50, and n = 500, n = 1000 or n = 2000. 



The solution of this equation, taking into account that y(0) = -k, is 
y(t) = 1 - (N - l)^- 1 (Jn - -) + - ■ e at y 
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where (3 denotes - - c H s n — . By imposing stationarity ^ = 0, we find the 

stationary solution 1, meaning that eventually all nodes will see d. 

In order to evaluate how closely y(t) models coverage, we use the traces from 
the simulations executed for Sec. 5.1. At every round, the nodes that carry a 
replica of d are identified, and a record of the nodes that have seen d since 
it was published is kept. Fig. 8 presents the coverage measured for four sets of 
experiments, each set with a different value for n. As n increases, a newly inserted 
item requires more time to cover the whole network. This is due to having more 
competition from other items to create replicas in the limited space available, as 
was previously shown in Fig. 7. However, as predicted by the stationary solution, 
in all cases the coverage eventually reaches 1. As shown in Fig. 8, the solution y(t) 
models the behaviour observed in simulations, falling nicely within the standard 
deviation of the simulation results. 

6 Conclusions 

In this paper, we have demonstrated that it is possible to model a gossip protocol 
through a rigorous probabilistic analysis of the state transitions of a pair of nodes 
engaged in the gossip. We have shown, through an extensive simulation study, 
that the dissemination of a data item can be faithfully reproduced by the model. 
Having an accurate model of node interactions, we have been able to carry out 
the following: 

— After finding precise expressions for the probabilities involved in the model, 
we provide a simplified version of the transition probabilities. These sim- 
plified, yet accurate, expressions can be easily computed, allowing us to 
simulate the dissemination of an item without the complexity of executing 
the actual shuffle protocol. These simulations use very little state (only some 
parameters and variables, as opposed to maintaining a cache) and can be 
executed in a fraction of the time required to run the protocol. 

— The model reveals relationships between the parameters of the system. Armed 
with this knowledge, we successfully optimize one of the parameters (the 
size of the exchange buffer) to obtain the fastest convergence of the observed 
properties. 

— Under the assumption of full connectivity, we are able to use the transition 
probabilities to model the properties of the dissemination of a generic item. 
Each property is ultimately expressed as a formula which is shown to display 
the same behaviour as the average behaviour of the protocol, verifying the 
validity of the model. 

While gossip protocols are easy to understand, even for a simple push/pull pro- 
tocol, the interactions between nodes are unexpectedly complex. Understand- 
ing these interactions provides insight into the mechanics behind the emergent 
behaviour observed in gossip protocols. We believe that understanding the me- 
chanics of gossiping is the key to optimizing (and even shaping) the emergent 
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properties that make gossiping appealing as communication paradigm for dis- 
tributed systems. 
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